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ABSTRACT 

Since 1988, Western Wi scons in Technical Col lege 
(WWTC) has taken a leadership role in integrating instituti onal 
effectiveness activities within the Wisconsin Technical College 
System (WTCS) . In December 1992, the WTCS Administrators Association 
and the WTCS State Board agreed to work with the University of 
Michigan's Community College Consortium to develop an institutional 
effectiveness model that would provide a valid and consistent means 
of evaluating the performance of each college and the system as a 
whole. The statewide effectiveness modeling project sought to 
identify core indicators that could be used by WTCS colleges to 
respond to accountability initiatives, accreditation requirements, 
federal reporting requirements, and quality management initiatives. 
In its adaptation of the statewide model, WWTC took the following 
steps: (1) representatives were selected to attend a state conference 
on institutional effectiveness; (2) an institutional effectiveness 
committee was formed; (3) the committee adapted the WTCS model by 
determining whether each core indicator in the model applied to the 
mission of WWTC and whether it could be measured using the resources 
available at WWTC; and (4) measurable factors of institutional 
effectiveness were identified for each core indicator in the 
categories of student achievement and satisfaction; organizational 
quality, harmony, and efficiency; public perception and satisfaction; 
and employer satisfaction. While this model helps detail the WTCS ' s 
commitment to its mission and purpose, WWTC has found that the Deming 
Plan-Do-Study-Act cycle of improvement provides the methodology for 
continuous improvement. (KP) 
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Integrating a Statewide Effectiveness Model with 
Quality Improvement Methods at Western Wisconsin 
Technical College 

Introduction 
The College 

Western Wisconsin Technical College (WWTC) is located in La Crosse, 
Wisconsin. The college serves approximately 7,500 credit students and 20,000 total 
students. The district is primarily rural with La Crosse being the major city. The 
eleven-county district contains approximately 220,000 residents. The college offers 
sixty programs in health occupations, business, industrial, agriculture, graphics and 
home economics fields. It is one of sixteen colleges in the V^consin Technical College 
System (WTCS). 

The Philoso-phy 

Most technical colleges are working on institutional effectiveness models and 
assessment plans. It is important that as these processes are put in place, they allow us 
to provide not only a record of past performance but also information for 
decision-making and improvement. Western Wisconsin Technical College is currently 
working on several activities in this area. They include: 

A) Taking a leadership role to integrate institutional effectiveness activities 
within the Wisconsin Technical College System, as well ?.s to integrate activities locally 
within the college among the different departments and divisions. 

B) Focus on the development of information that can be used for internal 
decision-making and not primarily to satisfy outside agencies. 

C) Encourage the involvement of the president, board of directors, 
administration, faculty and staff in the development and use of effectiveness measures. 

D) Focus not only on changing the way data is collected, but on improving the 
interpretation and use of data within the college. 

E) Making an effort to refine and improve existing data collection processes 
rather than create new processes or procedures unless absolutely required. 

The Initiative 

In 1988, Western Wisconsin Technical College made a commitment to 
implement quality management and a quality improvement process at the college. 
This new management philosophy is based on the works of Edwards Deming and 
focuses on systems development and systems thinking. It includes greater emphasis on 
communication and participation by faculty and staff in college-wide decision-making. 
This emphasis on quality improvement provided the basis for a very successful 
accreditation visit in 1991 by the North Central Association of Colleges and Schools 
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(NCA). The college was applauded for its successful. approach to identifying and 
measuring institutional effectiveness indicators. In spite of our early success in this 
area, there was a realization that new mandates from NCA relating to identifying and 
measuring institutional effectiveness and student academic achievement would 
challenge the college to more closely examine these issues. It also led the college to 
take a leadership position in developing an institutional effectiveness model for the 
State of Wisconsin. This model was developed for use by all sixteen colleges in the 
Wisconsin Technical College System. 

Development of the Wisconsin Technical College System Model 

In December of 1992, the WTCS Administrators Association and the WTCS 
State Board unanimously agreed to work with the Community College Consortium at 
the University of Michigan to develop an institutional effectiveness model specifically 
designed for the Wisconsin Technical Colleges. The main objective of this project was 
to develop a comprehensive effectiveness model for adaptation within the context of 
the individual colleges. 

The initiative was not intended to supplant the work that many of the colleges 
already have in place to measure their effectiveness, but rather to give the Wisconsin 
Technical College System some valid and consistent means of evaluating the 
performance of each individual college and the performance of the System. It was 
agreed, therefore, that the model to be developed would take into account existing 
efforts and would be used by the individual colleges in ways deemed most appropriate 
by each college. It was also agreed that the project would be undertaken 
simultaneously with other System-wide projects, such as the WTCS visioning project, 
to reaffirm the WTCS mission and goals. While this and other projects were 
undertaken independently, a great deal of cross-checking and communication took 
place to ensure that the final products would be complementary and consistent. 

In essence, the effectiveness modeling project is a System-wide effort to identify 
core indicators that can be used locally by WTCS Colleges to respond to 
accountability initiatives, NCA recommendations, federal reporting requirements, 
quality management initiatives, and so forth. The Wisconsin Technical College 
System, in taking this proactive step, is one of the first state-wide systems in the 
country to voluntarily embark on this type of initiative. 

WTCS Project Objectives: 

The WTCS effectiveness modeling project included eight specific objectives. 
Each of these objectives is described below: 

1. Examine existing models and systems (procedures, policies, and practices) for 
gathering information related to institutional effectiveness in the technical colleges. 
The project was designed to answer four questions: 
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a. What effectiveness models or practices are currently used in the technical 
colleges? 

b. What criteria do these models emphasize and for what purpose are they 
used? 

c. What effectiveness indicators are currently used in the technical colleges 
that require routine data collection? 

d. What effectiveness indicators are under consideration, require further 
consideration, or must be developed? 

2. Identify the types of published information (research reports, system data 
fields, accreditation reports, etc.) routinely available in the System Office and 
individual colleges that apply to the assessment of effectiveness. 

3. Appoint a task force comprised of key System Office staff and College staff 
(presidents, chief academic officers, chief student personnel officers, finance officers, 
faculty, support services personnel, etc.) holding specific interests that need to be 
addressed in the design and implementation of an effectiveness model. This task force 
will meet regularly to guide the project and to review the activities and findings of the 
consultants. 

4. Develop a matrix of key groups (on- and off-campus) holding specific 
interests in effectiveness. Employ a variety of information gathering techniques 
(document review, surveys, focus group meetings, and individual interviews) to 
determine: a) key institutional characteristics or performance outcomes which define 
effectiveness for each group and b) indicators and data gathering methods 
recommended for measuring effectiveness. 

5. Develop and test a preliminary effectiveness model based on indicators 
identified by different groups (via the activities in Objective 4) and distribute to 
representative groups in the technical colleges for review and revision. 

6. Revise the preliminary effectiveness model using feedback from different 
groups. 

7. Distribute the approved effectiveness model to representative groups 
throughout the system. 



8. Conduct a series of workshops throughout the state to: a) familiarize faculty 
and s'aff with the effectiveness model and b) provide training in methods and 
procedures for data collection to implement the model. 

Process and Procedures 

A task force, formed in the initial stages of the project, provided a natural 
communication link to the various subgroups of the Administrators Association. The 
task force met six times between January and August 1993 to develop a system-wide 
definition for institutional effectiveness, to provide input and direction for project 
activities, and to review the consultants' findings. In addition, task force members 
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arranged for, and in many cases were in attendance at, meetings held with different 
groups in the technical colleges. 



One of the guiding principles established by the task force in the earlv states of 
the project was that externally dictated indicators of effectiveness (e.g. NCA 
accredivation guidelines, federal and state legislation, etc.) while very important, 
should -iOt be the starting point for identifying effectiveness indicators that were 
critical for the Wisconsin Technical Colleges. "Going directly to the well" and getting 
input from representative groups about what is important in effectiveness seemed to be 
the right place to start. The first step, therefore, was to develop a working definition of 
effectiveness that could be used by the colleges and to solicit input from focus groups 
representing a wide variety of perspectives. Following a series of ten focus group 
meetings involving administrators, faculty, students, support staff., and board members, 
the consultants compiled an inventory of over 120 potential effectiveness indicators 
that was then circulated among representatives of functional areas, the Administrators 
Association, the District Boards Association, and student groups. 

Over 120 individuals from these groups ranked the importance of the indicators 
and identified the ten most important. From this feedback, the effectiveness indicators 
were rank ordered. Included in the process were: 

• district directors/presidents 

• ■ academic officers 

• student services officers 

• financial officers 

• instructors 

• students 

• planning/research/evaluation officers 

• economic development officers 

• technical reporting specialists 

• system administrators 

• support staff 

• information system directors 

• marketing officers 

• campus deans 

• trustees 

As the next step in the process, the consultants cross-checked the top-ranked 
indicators (i.e. "core indicators") against three "information banks" to ensure 
comprehensiveness and consistency: 1) the WTCS mission statement and 
effectiveness definition, 2) NCA student achievement guidelines, and 3) federal 
legislation. 
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Western Wisconsin Technical College Progression Toward 
Institutional Effectiveness 

WWTC Selects Representatives to Attend State Conference 

College representatives to attend a state conference on institutional effectiveness 
were chosen on the basis of their expressed interest in the pursuit of institutional 
effectiveness and their past record of achievement and service to the college. The 
membership of the group was also designed to include representatives from several of 
the major units which would later be responsible for the implementation of 
institutional effectiveness strategies. The representatives included faculty members 
from the Human Services division, the Business Education division the General 
Education division, the Counseling and Admissions Supervisor from Student Services, 
the WWTC North Central Association Coordinator, the Director of Planning, 
Evaluation, and Research, and the v ice President of Instruction. 

WWTC Forms Institutional Effectiveness Committee 

Through a series of discussions following the state conference, the 
representatives decided to create a formal institutional effectiveness committee for the 
purpose of creating and implementing a model at WWTC In addition to the state 
conference representatives, the committee was enlarged to include faculty from the 
General Education division, a clerical representative, the Curriculum Specialist, the 
Quality improvement Coordinator, an instructional support representative, and the 
faculty member who served on the WTCS Institutional Effectiveness Committee, 

While the size of the committee was large, it was determined that input from 
each of the major units on campus was needed to create a model which would meet the 
needs of the college. The charge of the committee was to identify measures, develop a 
model, and formulate specific methods by which institutional effectiveness could be 
and would be measured. The committee had authority to determine how the strategies 
would be implemented. Therefore, input from affected groups was an integral part of 
the development process. 

Institutional Effectiveness Committee Adapts State Model 

The WWTC Institutional Effectiveness Committee decided very early in the 
process to review and adopt, with modifications, the WTCS model. The adaptation of 
the state model to meet the needs of WWTC required a systematic approach which 
included several stages. The committee first reviewed WWTCs mission and vision 
statements to determine the main goals of the college. Next, the committee revisited 
WWTCs long-range strategic plans to determine if these plans were compatible with 
the mission and vision statements. The strategic plans were also evaluated to 
formulate measures of WWTCs progress toward specific objectives. 

Following the review process, the committee began to identify the core indicators 
of institutional effectiveness. Although the task initially seemed relatively easy, of 
primary concern to the committee were two questions: "Does the indicator apply to 
the mission of WWTC?" and "Can the indicator be measured using the resources 
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available at WWTC?" For each core indicator, an affirmative answer was required for 
both quescions. As a result, some of the indicators developed at the state level were 
found to be inappropriate for the needs of WWTC. Others could be adequately 
measured to report college efforts. 

Core Indicators of Institutional Effectiveness 

The core indicators of institutional effectiveness at WWTC fall into four 
categories: student achievement and satisfaction; organization quality, harmony, and 
efficiency; public perception and satisfaction; and employer satisfaction. The 
relationship of each of the core indicators to the others and co the larger concept of 
institutional effectiveness is shown by Figure 1. For each of the core indicators, the 
committee identified measurable factors of institutional effectiveness. The relationship 
of the specific factors to the core indicator, student achievement and satisfaction, is 
shown in Figure 2. While there may be other factors related to student achievement 
and satisfaction, the committee chose only those factors which could be measured with 
the resources available at WWTC. In addition, the measures of institutional 
effectiveness were summative evaluations, most of which could be expressed in 
quantifiable terms rather than through anecdotal statements. 

During the discussions about the core indicators and specific factors of 
institutional effectiveness, it was revealed that much of the data needed was already 
being collected on a regular basis. WWTC reports course completion rates, grades, 
graduation rates, scores on licensure exams, employment placement rates, etc. In 
addition, WWTC uses questionnaires and interviews to measure student achievement 
of educational goals, student and employer satisfaction with programs, and student 
satisfaction with services. Finally, factors such as the placement of new students at 
appropriate educational levels can be inferred from the measurement of other related 
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Figure 1 
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factors. The institutional effectiveness committee decided to avoid collecting and 
disseminating more data, and to eliminate the collection of useless data. The 
committee chose instead to focus its efforts on using existing data to provide 
information for use in the pursuit of continuous quality improvement. 



Continuous Improvement 

Western Wisconsin Technical College began its journey implementing 
continuous improvement in the late 1980's. Institutional effectiveness and continuous 
improvement are complementary. In essence, institutional effectiveness details the 
organization's commitment to its mission or purpose and continuous improvement 
provides the methodology. Dr. W. Edwards Deming taught that goals without 
measures are often arbitrary objectives that may be unattainable. Such objectives can 
suboptimize the organization and often have no relation to the organisation's mission. 
Institutional effectiveness provides an. organization with the plan to be t f fective and 
move towards attaining its mission. Inherent in the plan are the measures to insure 
that the plan is not only being implemented, but is the right plan. 

The Deming Cycle for Improvement, often referred to as Plan-Do-Study-Act 
(PDSA), provides a framework for process improvement. In the Plan stage, a change 
or plan for improvement is initiated. In the Do portion of the process the plan is 
carried cut on a small scale. During the Study stage, the results are analyzed. In the 
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Act portion of the process, the change is adopted, abandoned, or the PDSA stage is 
repeated. 

WWTCs Institutional Effectiveness Committee adapted PDSA to provide 
analysis for each of the core indicators. A worksheet was developed (Figure 3) that 
aligned the Plan stage with each Core Indicator, the Do stage with the question "How 
is it Measured?", the Study stage with the question "How is it Used?" and the Act stage 
with Process Improvement. For example, the core indicator (Plan) "identify customer 
needs" would be measured (Do) by special surveys, needs assessments and advisory 
committees. It would be used (Study) for feedback to units for decision making, and 
the process improvement (Act) would be to organize requests around college needs. 
Applying the PDSA cycle provides a framework for continuous process improvement 
within the Institutional Effectiveness Model. 
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Future Efforts 

The committee developed a timeline of activities that will be implemented 
during the 1994-95 academic year (Figure 4). This timeline will bring the college to 
the completion of the initial phase of the project. 



Institutional Effectiveness Focus 1994-95 



Workshop - May 23 
Drift developed of A*^$ment Plan. 

Workshop - June 23 
Refinement of fail iruervice packet. 

Model Workshop - Sept. 21 
Focus on indicators 
and development of draft. 

Model Workshop - Oct. 19 
Draft of Assessment Plan sent 
for NCA approval. 

Model Workshop • Nov. 16 
Refine tools, activities* and 
responsibilities. 




Assessment Plan material presented 
^to Administration. 

Present Assessment Plan to 
Faculty/Staff ar fall inservice 



Meet with various WWTC 
committee . 10 oOtain 
overlapping information. 



Fall survey results distributed 
and further feedback acquired 
from individual interviews. 

Assessment Plan material 
^presented to Administration. 

Present Assessment Plan material 
to Faculty/Staff ac inservice. 

Final Assessment Plan presented 
to Administration. 

Assessment Plan 
presented to Board 
tor approval and 
then to NCA. 
Projected implementation - August 199S, 



Figure 4 
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"In the future, the College will continue to assess student achievement by developing a 
comprehensive assessment plan." Therefore, the Institutional Effectiveness committee 
selected the core indicators dealing with student achievement as its high priority goal 
for demonstrating to NCA that WWTC is indeed following through on this 
assignment in a timely manner. 

Figures 1-3 graphics by Reva Heifetz, Quality Coordinator, Planning, Evaluation 
& Research Division, Western Wisconsin Technical College 

Figure 4 graphics by Barbara Fischer, Graphics Faculty, Industrial Technologies 
Division, Western Wisconsin Technical College 
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